In recent years, researches on reinforcement Learning (RL) have focused on bridging the gap between adaptive optimal control and bio-inspired Learning techniques. Neural network reinforcement Learning (NNRL) is among the most popular algorithms in the RL framework. The advantage of using neural networks enables the RL to search for optimal policies more efficiently in several real-life applications. Although many surveys investigated general RL, no survey is specifically dedicated to the combination of artificial neural networks and RL. This paper therefore describes the state of the art of NNRL algorithms, with a focus on robotics applications. In this paper, a comprehensive survey is started with a discussion on the concepts of RL. Then, a review of several different NNRL algorithms is presented. Afterwards, the performances of different NNRL algorithms are evaluated and compared in Learning prediction and Learning control tasks from an empirical aspect and the paper concludes with a discussion on open issues.

We address the throughput maximization problem for downlink transmission in DF-relay-assisted cognitive radio networks (CRNs) based on simultaneous wireless information and power transfer (SWIPT) capability. In this envisioned network, multiple-input multiple-output (MIMO) relay and secondary user (SU) equipment are designed to handle both radio frequency (RF) signal energy harvesting and SWIPT functional tasks. Additionally, the cognitive base station (CBS) communicates with the SU only via the MIMO relay. Based on the considered network model, several combined constraints of the main problem complicate the solution. Therefore, in this paper, we apply heuristic guidelines within the convex optimization framework to handle this complexity. First, consider the problem of maximizing throughput on both sides of the relay separately. Second, each side progresses to solve the complex problem optimally by adopting strategies for solving sub-problems. Finally, these optimal solutions are synthesized by proposing a heuristic iterative power allocation algorithm that satisfies the combinatorial constraints with short convergence times. The performance of the optimal proposed algorithm (OPA) is evaluated against benchmark algorithms via numerical results on optimality, convergence time, constraints’ compliance, and imperfect channel state information (CSI) on the CBS-PU link.

Face recognition from digital images is used for surveillance and authentication in cities, organizations, and personal devices. Internet of Things (IoT)-powered face recognition systems use multiple sensors and one or more servers to process data. All sensor data from initial methods was sent to the central server for processing, raising concerns about sensitive data disclosure. The main concern was that all data from all sectors that could contain confidential information was placed in a central server. Federated Learning can solve this problem by using several local model training servers for each region and a central aggregation server to form a global model in IoT networks. This article presents a novel approach to optimize data transfer and convergence time in federated Learning for a face recognition task using Non-dominated Sorting Genetic Algorithm II (NSGA II). The aim of the study is to balance the trade-off between training time and model accuracy in a federated Learning environment. The results demonstrate the effectiveness of the proposed approach in reducing data transfer and convergence time, leading to improved performance in face recognition accuracy. This research provides insights for researchers and practitioners to enhance the efficiency of federated Learning in real-world applications.

An important problem for the WiMAX networks is how to provide a guaranteed quality of service for applications. A key aspect of this problem is how BSs should share bandwidth capacity between different classes of traffic. This decision needs to be made for each incoming packet, and is known as the packet scheduling problem.A major challenge in packet scheduling is that the behavior of each traffic class may not be known in advance, and can vary dynamically.In this paper, we describe how we have modeled the packet scheduling problem as an application for reinforcement Learning (RL). We demonstrate how our RL approach can learn scheduling policies that satisfy the quality of service requirements of multiple traffic classes under a variety of conditions. The proposed solution has been designed to have an ability to accommodate integrated traffic in the networks with effective scheduling schemes. A series of simulation experiments have been carried out to evaluate the performance of the proposed scheduling algorithm. The results reveal that the proposed solution performs effectively to the integrated traffic composed of messages with or without time constraints and achieves proportional fairness among different types of traffic.

Social networks are one of the types of complex networks. Identifying communities in social networks is an effective way to use their information, for which several algorithms have been presented so far. In this paper, novel algorithms are designed, in which a Learning automaton is attached to each node; The number of actions of Learning automata is fixed and equal to the estimate of the number of network communities. At each step, each of the Learning automata chooses an action from its set of actions. Choosing any of these actions means assigning the label of that community to the node. The action chosen by each automaton is evaluated based on the chosen actions of its neighbors ((local attention) and/or communities detected by the entire method (global screening). The result of the evaluation leads to generate rewards or punish signal for the automata. By receiving a reward, the probability of re-choosing the chosen action by the automaton, or the community label, increases, and otherwise, by receiving a fine, the probability of this action decreases. By repeating the algorithm, the optimal action is determined as long as no change occurs in the selected label of the corresponding automata of each node with more iterations, and as a result, the optimal communities are determined as the output of the algorithm. The comparison of the results of the experiments shows the effectiveness of the proposed methods in comparison with the previous methods.